Papers by Farima Fatahi Bayat
Enhancing Language Model Factuality via Activation-Based Confidence Calibration and Guided Decoding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to calibrate language models are limited in inference-time efficiency or fail to provide informative signals. |
| Approach: | They propose an activation-based calibration method, ActCab, which trains a linear layer on top of the LM’s last-layer activations. |
| Outcome: | The proposed method improves on five popular QA benchmarks and reduces the average expected calibration error (ECE) score by up to 39%. |
FLEEK: Factual Error Detection and Correction with Evidence Retrieved from External Knowledge (2023.emnlp-demo)
Copied to clipboard
Farima Fatahi Bayat, Kun Qian, Benjamin Han, Yisi Sang, Anton Belyy, Samira Khorshidi, Fei Wu, Ihab Ilyas, Yunyao Li
| Challenge: | Existing large language models (LLMs) have a tendency to hallucinate and provide creative and fluent responses that are not factually accurate. |
| Approach: | They propose a tool that automatically extracts factual claims from text, gathers evidence from external knowledge sources, evaluates the factuality of each claim, and suggests revisions for identified errors. |
| Outcome: | The proposed tool detects errors in text and evaluates their factuality and suggests revisions based on the collected evidence. |
FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation (2025.acl-long)
Copied to clipboard
| Challenge: | Language models (LMs) generate false or unverifiable content, often known as hallucination, despite ongoing efforts to enhance their factuality. |
| Approach: | They propose a tool that measures LMs’ factuality in real-world user interactions by evaluating their factual accuracy and categorizing content units as Supported, Unsupported, or Undecidable based on Web-retrieved evidence. |
| Outcome: | The proposed evaluation pipeline measures language models’ factuality in real-world user interactions. |
From Proof to Program: Characterizing Tool-Induced Reasoning Hallucinations in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Tool-augmented Language Models can invoke external tools to solve problems beyond their parametric capacity. |
| Approach: | They propose a preference-optimization-based framework that realigns TaLMs to use tool outputs as assistive evidence. |
| Outcome: | The proposed framework improves accuracy and reasoning depth under tool use. |
Logit Arithmetic Elicits Long Reasoning Capabilities Without Training (2026.findings-acl)
Copied to clipboard
Yunxiang Zhang, Muhammad Khalifa, Lechen Zhang, Xin Liu, Ayoung Lee, Xinliang Frederick Zhang, Farima Fatahi Bayat, Lu Wang
| Challenge: | Large reasoning models exhibit long chain-of-thought reasoning with complex strategies such as backtracking and self-verification, yet, these capabilities typically require resource-intensive post-training. |
| Approach: | They propose a decoding-time approach which transfers long chain-of-thought reasoning capabilities from a substantially smaller reasoning guider to a large non-reasoning target. |
| Outcome: | The proposed method improves performance over a model 21x smaller than the target model by 21.5% and 24.2% over the model. |
Enhanced Language Model Truthfulness with Learnable Intervention and Uncertainty Expression (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) generate long-form and coherent text, yet they often hallucinate facts, which undermines their reliability. |
| Approach: | They propose a Learnable Intervention method for Truthfulness Optimization that automatically identifies the optimal intervention intensity tailored to each query context. |
| Outcome: | Experiments on multiple LLMs and question-answering datasets show that LITO improves truthfulness while preserving task accuracy. |
CompactIE: Compact Facts in Open Information Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Despite advances in open information extraction, many systems focus on covering more information over compactness of constituents. |
| Approach: | They propose a neural OpenIE system that produces compact extractions with overlapping constituents by using a pipelined approach. |
| Outcome: | The proposed system produces 1.5x-2x more compact extractions than previous systems, with high precision, establishing a new state-of-the-art in OpenIE. |